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Abstract. Physics homework often 

boils down to solving end-of-chapter 
quantitative problems. For targeting 
different learning goals of physics 
education, different types of homework 

are needed. The aim of this research was to 
compare the effectiveness of simulation- 
based, video-based, and paper-and-pencil 
homework in developing an understanding 
about Newton's laws and forming positive 
attitudes towards physics homework. 

150 first-year students from the Faculty 

of Chemical Engineering and Technology 
at the University of Zagreb (Croatia) were 
randomly assigned to one of the three 
above-mentioned homework approaches. 
After students had lectures and seminars 
on Newton's laws, they were administered 
a pre-test. In the next three weeks, the 
students completed three homework 
assignments on Newton's laws, after which 
they completed a post-test. For students 
from all three homework approaches a 
substantial improvement in conceptual 
understanding has been observed. 
Although the three approaches proved 

to be equally effective when it comes to 
developing understanding, the simulation- 
based approach was found to be superior 
when it comes to developing positive 
attitudes towards physics homework. If one 
controls for target knowledge, the modality 
of the homework assignment does not 
affect cognitive outcomes, but it does affect 
students’ attitudes towards homework. 
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Introduction 


One of the most important roles of the physics teacher is to design en- 
vironments which are very effective in activating, supporting, and sustaining 
those cognitive processes that are most relevant for students’ learning about 
a given physics topic. Thereby, the teacher is expected to prepare produc- 
tive learning environments not only for the classroom context but also for 
the homework context. In fact, learning physics is not limited to the physics 
classroom — much of physics learning may happen through the completion of 
homework. An important disadvantage of homework over classroom learning 
may be related to the fact that interaction with the teacher is largely missing. 
However, at the same time, a potential advantage of homework learning is 
that there are fewer time constraints and learners are in a better position to 
control the pace of their own learning. 

According to the model proposed by Trautwein et al. (2006), the effec- 
tiveness of homework learning is affected not only by the characteristics of 
the learning environment but also by students’ characteristics, homework 
motivation, parental behavior and homework behavior. Here the learning 
environment factor is mainly related to the quality and quantity of the home- 
work tasks, as well as to the teacher's feedback about students’ homework 
learning. How well the affordances provided by the learning environment are 
valorized, depends largely on the mere student, mainly on her/his cognitive 
ability and conscientiousness. However, for the relevant cognitive resources to 
be activated it is important that the student is motivated to do the homework. 
Concretely, she/he must perceive the homework activities as valuable and 
surmountable. This motivational aspect is important to start homework learn- 
ing, as well as to persist in completing homework activities. The mere learning 
process occurs through an interaction between the learner and the learning 
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environment. For the homework context, the main aspects of the learning environment are the characteristics of 
the homework tasks and parental behavior (e.g., monitoring homework completion, quality, and quantity of help). 
If this interaction between the learner, homework tasks (i.e., learning materials) and parents results in relevant and 
intensive cognitive processes, significant learning occurs. According to Sweller (1994), there are three mutually 
additive types of cognitive processes: relevant (i.e., processes that contribute to the construction of target knowl- 
edge), irrelevant (i.e., cognitive activities not related to constructing target knowledge) and intrinsic (i.e., related 
to the inherent complexity of the task). For most effective learning, a relevant cognitive load has to be fostered, 
irrelevant load minimized, and intrinsic load has to be adjusted (e.g., optimizing task structure) (Paas et al., 2003). 

It is very useful to realize that the only factor from Trautwein et al’s (2006) model which can be directly ma- 
nipulated by the teacher is the learning environment. Concretely, teachers may significantly influence the quality of 
homework learning experiences by carefully choosing homework tasks. In practice, conventional physics homework 
boils down to answering a few end-of-chapter questions that assess factual or conceptual knowledge, followed by 
the requirement to solve some quantitative problems. Thereby, often these questions and problems are chosenina 
random fashion, instead of being carefully selected to allow for an optimal upgrade of competencies developed in 
the science classroom (Me&i¢ et al., 2022). In addition, the typically assigned end-of-chapter questions and quanti- 
tative problems do not optimally reflect the spectrum of learning goals we tend to achieve in modern-day science 
education, such as goals related to science inquiry skills or goals related to developing positive attitudes towards 
science. Here attitudes towards science are defined as “feelings, beliefs and values held about an object that may be 
the enterprise of science, school science, the impact of science on society or scientists themselves” (Osborne, 2003, p. 
1053). Some of the components of the attitude towards science construct which are mentioned in earlier research 
are: motivation towards science, self-esteem in science, perception of achievement in science and perception of 
science classroom environment (Osborne et al., 2003). It is useful to distinguish between attitudes towards science 
and attitudes towards school science. Attitudes towards school science are the “product of students’ experience of 
school science” (Osborne, 2003, p. 1055), and it is exactly this construct which largely influences students’ career 
choices and their readiness for life-long learning of science. Considering that the number of students who choose 
careers in physics is continually decreasing and the societal needs for physicists are increasing (Kennedy et al., 
2014; Oon & Subamaniam, 2011; Pronovost et al., 2016; Roach & Sauermann, 2017), it is very important to find ef- 
fective ways for developing positive attitudes towards school physics in the students. When it comes to homework 
assignments, some alternatives to typical end-of-chapter tasks have to be found. 

Unlike end-of-chapter questions and problems, learning-by-inquiry activities are very well suited to target 
a wide spectrum of different learning goals, including the goal of developing a positive attitude towards school 
physics (Simsek & Kabapinar, 2010). However, learning-by-inquiry is known to be relatively time-consuming, which 
is one of the reasons why it is being avoided by some teachers, especially if they have only a small number of teach- 
ing hours at their disposal. Consequently, it may be useful to prepare learning-by-inquiry activities for the context 
of physics homework. Considering the fact that students generally love to learn with contemporary technologies 
(Li, 2007; Mesi¢ et al., 2022), a practical solution would be to situate inquiry-learning activities within the contexts 
of simulations or digital videos. A simulation can be defined as an “imitation of a real-world process or system over 
time” (Banks et al., 2010, p. 3). A particularly prominent feature of physics simulations is that it is very easy to change 
certain physical parameters and observe corresponding changes of the visually represented real-world processes 
or systems over time. In other words, the learner easily explores cause-and-effect relationships, which facilitates 
the development of functional mental models about physics phenomena (Dervic¢ et al., 2018). On some occasions, 
virtual experiments are believed to be an even better option compared to experiments with real equipment (De 
Jong et al., 2013). For example, this is the case when real setups are associated with considerable health risks or 
when manual data collection would be very time consuming. In addition, it is asserted that simulations provide 
implicit scaffolding (Moore et al., 2013; Roll et al., 2018), which is particularly important within the homework 
context because direct interaction with the teacher is missing. Concretely, well-designed simulations allow the 
learner to use the virtual equipment only in ways that support the achievement of learning goals — the number of 
degrees of freedom for using the virtual equipment is typically lower than for using real equipment. Consequently, 
irrelevant cognitive load is lower for virtual experiments, which means more working memory capacity for running 
relevant cognitive processes. However, which cognitive processes will be run largely depends on the tasks the 
students are expected to implement within the context of a simulation. The tasks should be designed to activate 
cognitive processes which are recognized as very important for learning a given physics topic. For example, within 
the context of Newton's first law, thinking about gradually decreasing friction is one of the key cognitive processes. 
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Generally, for developing conceptual understanding “predict-observe-explain” activities proved to be useful in 
earlier research (Karamustafaoglu & Mamlok-Naaman, 2015). 

To make sure that all students engage in fruitful cognitive processes, explicit scaffolding may be provided 
through carefully designed worksheets (Adams et al., 2015). These worksheets can include prompts that facilitate 
the planning and implementation of inquiry activities, as well as promote monitoring and self-evaluation of the 
learning process. 

Another technologically rich approach to physics homework would be to situate homework tasks within the 
context of videos (Laws et al., 2015). Videos may be used for purely informative purposes or for organizing learning 
by inquiry. The inquiry-variant of video-based homework may require the learner to analyze the video qualitatively 
or quantitatively to be in a position to answer certain questions that target key cognitive processes related to the 
given physics topic. Unlike interactive simulations, videos do not offer the opportunity for hands-on exploration 
of cause-and-effect relationships. However, videos may provide the students with the opportunity to observe and 
analyze physical phenomena to come to certain conclusions about relationships between physical quantities. 
On the other hand, there are also approaches in which the learner is expected to perform measurements from 
the videos on their own with the aim of setting up models of physical phenomena (e.g., in digital video analysis) 
(Brown & Cox, 2009). 

Whatever homework approach is used for learning about Newton's laws, it is important that the homework 
tasks provoke cognitive processes that are at the core of Newtonian mechanics. In mechanics, it is important for 
the students to learn the difference between acceleration and velocity, as well as the difference between the force 
and effects of force (Aviani et al., 2015; Erceg & Aviani, 2014). Force must be perceived as a measure of interaction 
between some physical entities. Thereby, students should realize that the forces of action and reaction must be of 
the same nature and that they never act on the same body (Knight, 2015). They also must realize that a change of 
the velocity vector is always associated with a non-zero resultant force and that objects may sustain their motion, 
even if there are no forces acting on them. Here it is also important that students overcome the misconception that 
static objects like tables cannot exert forces (Clement, 1993) and that they develop the ability to correctly identify 
forces, including the force of friction which is often not regarded as the force by students (Kizilcik et al., 2021). 


Research Aim 


There are not many studies about the relative effectiveness of simulation-based, video-based, and paper-and- 
pencil homework. In the study by MeSi¢ et al. (2022), simulation-based homework proved to be substantially more 
effective than paper-and-pencil homework when it comes to developing a conceptual understanding about gas 
laws in the upper-secondary school context. In a similar study by Simi¢ et al. (2022), simulation-based homework 
proved to be equally effective as video-based and paper-and-pencil homework when it comes to developing 
conceptual understanding about energy and work, but significantly more effective than any of these two ap- 
proaches when it comes to developing positive attitudes towards physics homework in upper-secondary school 
students. Therefore, one can say that the findings from earlier research on simulation-based physics homework are 
mixed and mostly situated within the upper-secondary school context. In addition, a significant limitation of the 
above-mentioned two studies is related to the fact that for the paper-and-pencil group, the homework questions 
were not carefully selected to target as similar as possible conceptual knowledge compared to questions from the 
simulation-based and video-based groups. 

The aim of this research was to compare the effectiveness of simulation-based (SB), video-based (VB) and 
paper-and-pencil (PP) physics homework when it comes to developing an understanding about Newton’s laws 
and forming positive attitudes towards physics homework in university students. Concretely, the research ques- 
tions were as follows: 

1. How effective are the SB, VB and PP approaches, when it comes to developing a conceptual understand- 
ing about Newton's laws in university students? 

2. How effective are the SB, VB and PP approaches, when it comes to forming a positive attitude toward 
physics homework in university students? 

Considering that students are generally fond of contemporary technologies (Li, 2007), it has been expected 
that simulation-based and video-based approaches are superior to the paper-and-pencil approach when it comes 
to developing positive attitudes towards physics homework. When it comes to the development of conceptual 
understanding, the potential advantage of the simulations is mainly related to the interactivity feature (Kalyuga, 
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2008), whereas a potential advantage of the video-based approach is that students receive more guidance (e.g., 
technical aspects of the given procedures were performed by the teacher) which potentially leaves the students 
more time for thinking. Finally, a potential advantage of the paper-and-pencil approach is related to a more familiar 
format of the homework tasks and simpler task formulations (i.e., less irrelevant load, and less intrinsic cognitive 
load). However, for the paper-and-pencil approach, there is a lack of dynamic visualizations and interactivity, which 
may be an important disadvantage of this format, leading to less effective homework learning. 

The significance of this research is related to the fact that it describes in detail two alternative approaches to 
physics homework (i.e., simulation-based, and video-based physics homework) and their effects on some aspects 
of physics learning. In the current teaching practice, physics homework often fails to target a wide spectrum of 
learning goals (cognitive as well as affective ones) that are important for physics education. Concretely, students 
are typically asked to solve a few end-of-chapter, quantitative problems which potentially contribute to develop- 
ing students’ problem-solving skills, but fails to target some other, important learning goals, such as developing 
science inquiry skills or developing a positive attitude toward physics. Consequently, there is a need for research- 
based, alternative approaches to physics homework which could help the teachers to target a wider spectrum of 
learning goals. For example, by identifying homework approaches that are perceived positively by the students, 
we can expect more students to become ready for lifelong learning of physics and to choose a career in physics. 
On the other hand, having more young people who choose their career in sciences is very important for meet- 
ing the demands of technology-based societies (Maltese & Thai, 2011). In addition, comparing video-based and 
simulation-based homework might be a significant contribution to physics education literature, because some- 
times the students do not have at their disposal a certain simulation in their home environment, but the teacher 
can screen-record the simulation and ask them to analyze the corresponding video at home. Therefore, it might 
be useful to compare the effectiveness of simulation-based and corresponding video-based homework, across 
various physics topics and student populations. 


Research Methodology 
Research Design 


A pre-test - post-test true experimental study with three comparison groups has been conducted (Ary et al., 
2009). The research design is described in Table 1. 


Table 1 
Research Design 


Week 3 of semester 


2 hours of lectures and 
2 hours of seminars on 
Newton’s laws (same 
for all students) 


Week 4 of semester 


Random assignment 
of the students to the 
three homework ap- 
proaches 


Week 5 of semester 


Conceptual pre-test on 
Newton's laws 


Homework on New- 
ton’s first law 


Same lectures and 
seminars on other 
topics 


Week 6 of semester 


Homework on New- 
ton’s second law 


Same lectures and 
seminars on other 
topics 


Week 7 of semester 


Homework on New- 
ton’s third law 


Same lectures and 
seminars on other 
topics 


Week 8 of semester 


Conceptual post-test 
on Newton's laws 


Survey on attitudes 
towards school physics 


Same lectures and 
seminars on other 
topics 


The conceptual pre-test on Newton's laws has been conducted after all the students had the same lectures and 


seminar sessions on Newton's laws. This allowed us to associate the pre-test-post-test conceptual gain solely with 
the effect of homework learning. Considering that random assignment has been conducted and that the group 
sizes were relatively large, one can be pretty confident that the three student groups (SB, VB, PP) were comparable 
across all latent traits at the moment of entering the homework interventions (Ary et al., 2009). After week four 
the only systematic effort related to learning about Newton's laws was the physics homework. The homework 
was different for the SB, VB and PP groups. Consequently, potential between-group differences on the conceptual 
post-test and attitude survey may be validly related to differences between the three homework approaches. 
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The pre-test and post-test have been administered within regular seminar sessions, while the attitude survey 
has been administered as a Google Form and filled out by the students outside of university classes. For purposes 
of easier and more valid pre-test - post-test comparisons, the same conceptual test has been used as a pre-test and 
post-test. For all students, the lectures were given by the same teacher. All research activities from Table 1 were 
conducted between October 24" 2022 and December 4" 2022. 


Participants and Curriculum 


The target population consisted of first-year engineering students in introductory, two-semester physics 
courses. In this research, the student sample included 150 first-year students (mostly 19-year-olds) from the Faculty 
of Chemical Engineering and Technology at the University of Zagreb, Croatia. All participants gave their informed 
consent to participate in the study after being made aware of its purpose. They were assured that the principles 
of anonymity and confidentiality would be adhered to in the study. 

All 150 students who attended the Physics | seminar sessions in the fifth (pre-test) and eighth week (post-test) 
of the semester were included in the sample for drawing conclusions about the cognitive effects of the SB, VB and 
PP homework approaches. Out of these 150 students, 122 were female and 28 were male. On the other hand, the 
attitude survey has been filled out by 137 students (114 female, 23 male). The students were randomly assigned to 
one of the three homework approaches: simulation-based (n,, = 51), video-based (n,, = 48) and paper-and-pencil 
lee =51). 

In their first year of studies, students from the given sample enroll in the Physics | (fall semester) and Physics II 
(spring semester) courses. These courses include two teaching hours of lectures and two teaching hours of seminar 
sessions per week. In the Physics | course two hours of lectures and two hours of seminar sessions are devoted 
to learning about Newton’s laws. Lectures mostly focus on facts and mathematical procedures, while seminar 
sessions focus on solving numerical problems. Concretely, in the seminar sessions most of the time, the teaching 
assistant is modelling the problem-solving process to a class consisting of 30-40 students. Generally, the Physics | 
and Physics II curriculum include all the physics topics that are typically presented in widely known introductory 
physics textbooks, such as Physics for Scientists and Engineers by Serway and Jewett (2013). 


Description of Homework Approaches 


Here simulation-based homework boils down to learning physics by interacting with a simulation and complet- 
ing the corresponding worksheet prepared by the teacher. For the video-based homework, the only difference to 
simulation-based homework is related to the fact that students do not interact with the simulation on their own, 
but they learn by analyzing the screen-recorded version of the simulation that had been originally manipulated 
by the teacher. Again, nearly the same worksheet is completed as in the simulation-based group. Finally, for the 
paper-and-pencil group, the tasks resembled as much as possible the tasks from the simulation-based and video- 
based treatments. For example, the students were asked to perform a thought experiment instead of conduct- 
ing/observing a virtual experiment, or they were provided with the same experimental data (as was collected by 
students in SB) and asked to analyze it. 

It is important to note that the paper-and-pencil homework approach from this research cannot be considered 
as conventional physics homework, because its tasks are not typical end-of-chapter questions and problems. How- 
ever, unlike VB and SB homework, PP homework did not allow for the collection of measurements and observation 
of processes. In all three approaches, the activities mainly included identifying patterns and applying knowledge 
for predicting the outcome of certain processes. The VB and SB worksheets often included tasks related to the 
exploration of relationships, as well as predict-observe-explain activities. For purposes of providing a vivid insight 
into the similarities and differences between the three homework approaches, a complete, translated version of 
the materials for homework on Newton's second law is given at the following link: http://fizika.pmf.unsa.ba/wp- 
content/uploads/2023/02/Newton_2nd_law_worksheets.pdf 

Significant efforts have been invested in targeting the same key conceptual ideas in all three homework ap- 
proaches. To that end, paper-and-pencil homework questions were developed, only after the worksheets for SB 
and VB approaches had been completed. The prominent Newtonian ideas that were targeted through homework 
activities in all three groups are described in Table 2. 
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Table 2 


Prominent Newtonian Ideas Targeted through Homework Activities 


Newton’s first law 


Velocity of an object does not change if and only 
if the resultant force acting on the object is zero. 


An object may rest even if we apply a force on 
it (e.g., if there is a friction force of the same 
magnitude but opposite direction, acting on it). 


Any object tends to maintain its velocity (magni- 
tude as well as its direction); mass is the measure 
of inertia. 


Newton’s second law 


Non-zero net force (i.e., unbalanced forces) acting 
on an object changes the velocity of that object. 


Type of object's motion depends on the sum of 
all forces acting on that object; non-living objects 
also may exert a force and static friction forces 
can vary from zero to a maximum value. 


If net force and velocity vector have the same 
direction, at that moment velocity magnitude is 
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HOMEWORK 
(pp. 506-519) 


Newton’s third law 


When two objects interact, a pair of forces oc- 
curs; these forces are of the same nature and 
magnitude but act on different bodies and point 
in mutually opposite directions. 


Effects of force application do not depend only on 
the magnitude of a force but also on the mass on 
which the force is acting (e.g., in the collision of a 
truck and car, forces of the same magnitude act 
on both, but the deformation of the car is larger). 


increasing; if they are mutually opposite, mag- 
nitude decreases. 


Three Google classrooms have been created for the SB, VB and PP groups of students. Each Saturday morning, 
starting from the fifth week of the semester, the homework assignments were uploaded to the Google classrooms. 
The SB and VB students were provided with links to the simulations/videos, respectively, and they were also 
provided with corresponding worksheets. On the other hand, the PP students were only provided with a sheet 
containing questions and problems. All students have been asked to complete the homework before Sunday and 
upload the scans of their homework solutions to the Google classrooms. To motivate the students to invest more 
effort into completing homework, the 10% of students with the highest homework scores were promised to get 
some extra credits which could affect their final grade in Physics | course. On Sundays, the teacher uploaded the 
official homework solutions to the Google classrooms and asked the students to compare these official solutions to 
their own solutions, i.e., she encouraged them to perform self-reflection and to report time invested in homework 
completion. The teacher also scored all the submitted homework solutions; each submitted homework solution 
was assigned with a percentage score, based on the correctness of students’ reasoning and final solutions. 


Instruments 


For the research aim to be fulfilled, evidence on students’ conceptual understanding of Newton’s laws, as well 
as on their attitudes towards physics homework, had to be collected. According to Goldwater and Schalk (2016), 
conceptual understanding is relational knowledge about the core concepts in a domain and their interrelations. 
For the domain of mechanics, the core concepts are displacement, velocity, acceleration, mass and force. Some 
of the cognitive processes that may reflect conceptual understanding are: interpreting, comparing, identifying, 
predicting and explaining (Anderson & Krathwohl, 2001). 

Based on these theoretical considerations about the construct of conceptual understanding, as well as on the 
list of prominent Newtonian ideas from Table 2, 20 multiple-choice items with a single correct answer and three 
distracters were selected from existing literature or newly created. Concretely, 11 out of the 20 items were adapted 
(e.g., compared to original items only the number of distracters may have been changed) from earlier published 
literature, such as: Force Concept Inventory (Hestenes et al., 1992), Mechanics Baseline Test (Hestenes & Wells, 
1992), Force Body Diagram Test (Aviani et al., 2015), TIPERS (Hieggelke et al., 2015) and College Physics by Knight 
et al. (2015). The same 20 items were used for the pre-test and for the post-test. Ideas targeted by the individual 
items are briefly described in Table 3. 
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Table 3 


Brief Description of Ideas Targeted by the Conceptual Test 
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Item 1 
F,, <0 > v=const, context of 
smooth surface 


Source: Original 


Item 2 
v=const#0 > balanced forces 


Source: FCI (item 25) 


Item 3 

unbalanced forces > v#const; 
F 4. and v in same direction >v 
increases 

Source: FCI (item 26) 


Item 4 

unbalanced forces > véconst; 
F 4.and v opposite >v de- 
creases 

Source: FCI (item 27) 


Item 5 

v=0 > balanced forces 
non-living objects may exert a 
force 

Source: FBDT (item 1) 


Item 6 
v=const#0 > balanced forces; 
identifying forces 


Source: FBDT (item 3) 


Item 7 


v increasing > unbalanced forces; 


Fin the same direction as v 


nei 


Source: FBDT (item 4) 


Item 8 
Objects 1 and 2 interact, m,>m, 
>F,,=F,,, context of pushing off 


12° 21) 


Source: FCI (Item 28) 


Item 9 
Objects 1 and 2 interact, m,>m, 
DF =F, a,<a, 


Source: Original 


Item 10 

An object resting in a moving 
vehicle tends to maintain v, when a 
vehicle stops 

Source: Original 


Item 11 

Objects 1 and 2 interact, m,>m, 
> F,,.=F,,, context of collision 
Source: Original 


Item 12 
balanced forces > v=const 


Source: Original 


Item 13 

A force F with a direction opposite 
to v tends to decrease v 

Source: Original 


Item 14 
Objects 1 and 2 interact, m,>m, 
DF =F ,,7a,<a, 


Source: Original 


Item 15 
F__=0 > v=const, context of 


net 


Space 


Source: Original 


Item 16 
Linear increase of v > F=const#0 


Source: MBT (item 3) 


Item 17 
v=const#0 > balanced forces; 
context of elevator 


Source: FCI (item 17) 


Item 18 

v=0 while F, and F,, act on the 
object >F =F, not an action- 
reaction pair 

Source: Original 


Item 19 

Larger v generally does not imply 
larger F acting on the body 
Source: Knight et al. (2015) 


Item 20 

Objects 1 and 2 interact, m,>m, 
>F,,=F,,, context of vector 
representation 

Source: TIPERS (B3-QRT96) 


Since eleven items were adapted from widely accepted literature, and the other nine items covered mostly the 
same Newtonian ideas but situated in different contexts, it can be concluded that the given test may be used for 
drawing valid conclusions about university students’ conceptual understanding of Newton's laws. Next, reliability 
analyses for the conceptual test have been conducted. For item 18 the item-total correlation proved to be negative 
and consequently, this item has been removed from the scale. The Cronbach’s alpha of the 19-item scale amounted 
to .71 for the pre-test and .79 for the post-test. Both values indicate reliable test scores (Cohen et al., 2007). 

For measuring students’ attitudes towards physics homework, a shortened version of the attitude survey de- 
veloped by Me&i¢ et al. (2015) has been used; the same instrument has also been used in the homework research 
by Simi¢ et al. (2022). The attitude survey initially included six statements for which students were expected to 
express their level of agreement on a 4-point Likert scale (Table 4). 


Table 4 


Attitude toward Physics Homework Items 


Item 1 


The homework on Newton's laws made me 
interested to learn more about this topic. 


Item 2 


| feel that the homework on Newton's laws 
helped me to considerably improve my knowl- 
edge about Newton's laws. 


Item 3 


Although | invested efforts into the homework 
on Newton’s laws, it was difficult for me to 
complete it. 


Item 4 


While doing the homework on Newton’s laws, | 
have been learning physics with understanding. 


Item 5 


Learning about Newton’s laws boils down to 
memorizing facts and equations, and math- 


Item 6 


ematical solving of these equations. 


We should more often do homework like the 
one on Newton’s laws. 


AS 


https://doi.org/10.33225/jbse/23.22.506 


Journal of Baltic Science Education, Vol. 22, No. 3, 2023 


ISSN 1648-3898 /Print/ SIMULATION-BASED AND VIDEO-BASED APPROACHES TO DIVERSIFYING PHYSICS 
HOMEWORK 
ISSN 2538-71 38 /Online/ (pp. 506-519) 


Eventually, reliability analyses showed that Item 5 from the attitude survey was characterized by a negative 
item-total correlation and consequently it has been excluded from the attitude towards physics homework scale. 
The reliability of the final attitude scale was found to be .71 which is regarded as reliable, particularly for such a short 
scale as the one used in this research (Cohen et al., 2007). In addition, the students were also asked the following 
open-ended question: “What is your general impression about the Newton's laws homework?" 


Data Analysis 


For students who wrote the pre-test as well as the post-test, the answers of each student on each item were 
entered into SPSS. Thereby, the response option “a” has been coded as“1",“b"as“2",“c"as“3",and“d” as “4” Next, the 
answers have been recoded - correct answers were coded as “1” and incorrect as “0”. After the reliability analysis 
had been conducted, composite scores on the pre-test and post-test were calculated by simple summing of scores 
across individual items. Initially, the plan was to use the typical parametric tests for checking: 1) whether the pre- 
test-post-test gain was significant (paired t-test) 2) whether between-group differences on post-test were significant, 
after controlling for pre-test differences (ANCOVA). The most important assumptions for the classic paired t-test 
were met: Shapiro-Wilk test and Kolmogorov-Smirnov tests of normality were non-significant for pre-test-post-test 
differences within the three groups and there were no extreme outliers in the differences. However, for ANCOVA the 
statistical assumption of normal distribution of test scores within the groups was not met. Consequently, instead 
of classic ANCOVA, the WRS2 R package has been used to conduct a robust version of ANCOVA (Field, 2018; Mair & 
Wilcox, 2020). Considering that robust ANCOVA can be conducted for two groups at one time only, for the purpose 
of conducting ANCOVA two separate databases have been created: one included SB and VB data, and the other 
included SB and PP data. This made it possible to compare SB versus VB, and SB versus PP. 

When it comes to the attitude towards the homework part of the study, students’ Google form answers were 
exported to an SPSS data file. For the attitude towards homework items, the following answer coding has been 
used: 1-Fully disagree, 2 - Mostly disagree, 3 - Mostly agree, 4- Fully agree. Initially, items 3 and 5 were negatively 
keyed and consequently, they had to be reverse coded. After item 5 was removed due to negative item-total cor- 
relation, a composite score was calculated by summing the five positively keyed items. Because the assumptions 
of the classic ANOVA were not met (e.g., the variance in SB was much smaller than in the other two groups), a 
robust variant of one-way independent ANOVA with post-hoc tests, as described by Field (2018) has been used 
to analyze the significance of between-group differences on the attitude scale. In addition, chi-square tests have 
been conducted for checking for between-group differences on individual Likert-type items. All analyses have 
been performed in IBM SPSS 26 and R software, version 4.2.2. 


Research Results 


Descriptive results from the conceptual pre-test and post-test are reported in Table 5. The test scale ranged 
from 0 to 19 points. 


Table 5 
Mean Scores (M), Standard Deviations (SD) and Standard Errors (SE) at Pre-Test and Post-Test 


Simulation-based approach Video-based approach Paper-and-pencil approach 
(n,,=51) (n,,=48) (n,,=51) 
Pre-test M= 8.49; SD = 3.38; SE = 0.47 M= 7.77, SD = 3.33; SE = 0.48 M= 7.75; SD = 3.24; SE=0.45 
Post-test M= 14.29; SD = 3.75; SE = 0.52 M= 14.12; SD = 3.52; SE = 0.51 M= 14.25; SD = 3.19; SE = 0.45 


The pre-test-post-test difference for the VB approach proved to be statistically significant, t(47) =-11.88, p< .001, 
and represented an effect of d= 1.91. This is a large effect. In fact, the percentage of correct answers increased from 
40.9% to 74.3%. The pre-test-post-test differences for SB and PP approaches were also statistically significant; for 
SB t(50) = -11.98, p < .001 and for PP t(50) = -12.09, p < .001. For the SB approach, there was an effect of d= 1.71, 
and for the PP approach an effect of d = 2.00. In the SB group, the percentage of correct answers increased from 
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44.7% to 75.2%, and in the PP group from 40.8% to 75.0%. Next, the post-test differences between SB and PP were 
analyzed, while the pre-test scores were controlled (Table 6). 


Table 6 
Results of Robust ANCOVA for SB versus PP Comparison 


n,, Np M57 Mp, Lower Cl Upper Cl Statistic p value 
Pre-test = 4 20 18 -1.25 -5.30 2.80 -0.87 Al 
Pre-test = 5 22 19 -0.89 -4.57 2.78 -0.69 ot 
Pre-test = 7 31 28 -0.94 4.13 2.23 -0.84 A2 
Pre-test = 9 20 27 0.26 -2.85 3.38 0.24 19 
Pre-test = 12 12 12 1.25 -1.54 4.04 1.26 25 


The analysis of post-test differences between SB and VB, while controlling for pre-test scores, is presented in 
Table 7. 


Table 7 
Results of Robust ANCOVA for SB versus VB Comparison 


n,, ny M4” May Lower Cl Upper Cl Statistic p value 
Pre-test = 4 20 17 -0.04 -5.37 5.30 -0.02 99 
Pre-test = 5 22 24 -0.61 -5.02 3.80 -0.39 69 
Pre-test = 6 27 25 -1.24 -4.96 2.48 -0.95 35 
Pre-test = 9 20 24 0.56 -2.53 3.66 0.52 60 
Pre-test = 11 18 15 0.83 -1.19 2.85 1.17 .28 


Next, the between-group differences on the attitude scale were analyzed. The mean attitude scores in the 
SB, VB and PP groups were 17.89, 16.25 and 15.16, respectively (the theoretical maximum of the scale was 20). A 
bootstrapped version of robust ANOVA showed that there were statistically significant between-group differences 
on the attitude towards physics homework scale, F, = 19.34, p < .001. The corresponding effect size amounted to 
.51 which can be considered as large (Mair & Wilcox, 2020). In addition, the post-hoc tests showed that the SB ap- 
proach was superior to VB (difference in trimmed means = 1.30, p = .02), as well as to PP (difference in trimmed 
means = 2.47, p < .001). The VB approach proved to be superior to PP (difference in trimmed means = 1.18, p = .04). 
It should be noted that in all these comparisons the p values were adjusted for multiple testing (Mair & Wilcox, 
2020). Finally, chi-square tests were conducted to explore between-group differences on individual Likert-type 
items. Statistically significant between-group differences were observed for item 6 (x?(6) = 28.03, p < .001) and item 
4 (y?(6) = 24.94, p < .001) from the attitude survey. Figure 1 shows how students from different groups answered 
to these two items. 
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Figure 1 


Attitude Items with the Largest Between-Group Differences 
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Attitude 4 (learned with understanding) Attitude 6 (should more often have homework like this) 


When it comes to the open-ended question asking about the general impression, most students provided brief 
answers indicating a positive impression (e.g., “was OK’, “very good”). It may be most informative to quote selected 
answers, from different groups. For example, one student from the PP approach stated:"I liked the homework questions 
because they motivated me to think and for each question, we could answer something, even something wrong. Later, we 
could see where our reasoning, i.e., our perception of the situation went wrong.” One student from the SB group made the 
following statement:“/t made me explore Newton's laws in more detail and find out what they are all about. !am absolutely 
certain that now | better understand this topic.” Finally, one student from the VB group asserted the following: “More in- 
teresting than mere quantitative problem solving in the classroom and, | repeat it once again, better understanding through 
visualized situations.” However, some other students, mainly from the VB group, expressed also negative experiences, 
such as: “Some problem statements were not clearly specified”. Students from the VB group reported that they, on aver- 
age, invested 90 minutes in completing the homework, while in the SB group and PP group, the average homework 
time amounted to 80 minutes and 52 minutes, respectively. 


Discussion 


All three homework approaches resulted in statistically significant and large improvements in conceptual un- 
derstanding of Newton's laws. This finding may be explained by Trautwein et al‘s (2006) model of homework learning. 
Concretely, students solved the homework assignments after they had lectures and seminar sessions on Newton's laws. 
This means that they entered the homework-solving process at least with some basic foreknowledge. They were also 
extrinsically motivated to engage in the homework-solving process because extra credits were promised to students 
who achieve top homework scores (Cheng et al., 2004). In the VB and SB groups, the visually rich and/or interactive 
learning environment provided an additional source of (intrinsic) motivation. Even more important is that the home- 
work tasks were designed to reflect key Newtonian ideas, such as the ones presented in Table 2. Students’ answers 
from the attitude survey show that most students perceived the homework tasks as challenging and interesting, which 
motivated them to invest significant time and cognitive resources into the completion of homework. Thereby, much of 
the cognitive load was related to implementing relevant cognitive processes (e.g., more than 80% of students from all 
groups agreed with the statement that they learned about Newton's laws with understanding), which is key to effective 
learning (Sweller, 1994). Some students also lauded the fact that they could learn from the teacher's feedback, which 
is in line with earlier findings on the importance of feedback about homework (Tas et al., 2016; Xu, 2016). 

All three homework approaches proved to be equally effective when it comes to developing a conceptual under- 
standing about Newton's laws. The potential for generating relevant cognitive load was probably somewhat higher 
in SB and VB than in PP, because SB and VB allowed for interactivity and/or learning with multiple representations 
(Ainsworth, 2006; Kalyuga, 2008). However, in SB and VB the intrinsic and irrelevant load were higher too (e.g., due 
to more complex homework instructions and due to learning how to do homework with simulations or videos). For 
some SB and VB students, this certainly resulted in cognitive overload. Data from Table 6 suggest that this cognitive 
overload mostly occurred in SB students with low level of foreknowledge. The potential of the SB approach seems to 
be better valorized by students with a higher level of foreknowledge. 
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When it comes to the development of conceptual understanding, the results from this research are in line with 
the work by Simic et al. (2022) who found that simulation-based, video-based, and paper-and-pencil homework were 
equally effective in developing conceptual understanding about energy and work. Consequently, the findings by 
Simic et al. (2022) related to the equal effectiveness of simulation-based and corresponding video-based homework 
may be extended to university students and some other physics topics, such as Newton's laws. On the other hand, the 
results from this research seem to be not in line with the results by MeSic¢ et al. (2022) who found that the simulation- 
based homework was significantly more effective in developing students’ conceptual understanding about gas 
laws compared to paper-and-pencil homework. However, in that study, the paper-and-pencil homework included 
typical end-of-chapter problems, i.e., the key conceptual ideas were not carefully controlled across the homework ap- 
proaches. In addition, unlike the Newton's laws simulations, the gas laws simulation visualized abstract processes on 
the level of micro-particles which was more difficult to translate into paper-and-pencil homework. The effectiveness 
of simulation-based homework, when it comes to producing substantial learning gains, is in line with the findings by 
Adams et al. (2015). They showed that simulation-based homework which includes carefully scaffolded worksheets 
may help the students to achieve large learning gains, comparable to learning gains related to simulation use in the 
classroom setting. According to Wieman et al. (2010), a specific strength of simulation-based homework is that it al- 
lows the exploration of phenomena related to, but different from the ones observed in the class (e.g., high altitude 
breathlessness). The opportunity to analyze a wide spectrum of different, vivid phenomena is also a characteristic of 
the video-based physics homework. In the study by Laws et al. (2015), out-of-class learning with interactive videos 
resulted in a significantly increased conceptual understanding of Newton's laws. Similarly to videos from this research, 
the videos from the study by Laws et al. (2015) required the students to carefully observe physics phenomena and 
make predictions. This is consistent with de Araujo and Otten’s (2017) idea of improving the quality of video-based 
homework through the inclusion of interactive elements (e.g., applets, quizzes). They also point out the importance 
of systematically associating homework activities with specific learning goals. 

The simulation-based approach proved to be superior to the other two approaches when it comes to develop- 
ing a positive attitude toward physics homework. This result is in line with earlier research (Mesi¢ et al., 2022; Ronen & 
Eliahu, 1999; Simi¢ et al., 2022). In fact, most students like to learn with contemporary technologies (Li, 2007). Concretely, 
students praise simulations for providing them with vivid feedback about their own learning, which is particularly 
important in the homework context, where the teacher is absent (Ronen & Eliahu, 1999). Generally, students like to 
have an active role and to control the pace of their learning (Fouts & Myers, 1992; Kay & Edwards, 2012; Maltese & Thai, 
2011). Unlike in the SB approach, in the VB approach the students did not have the opportunity to interact on their 
own with the simulation. They were asked to stop the video at certain instants to answer some questions which means 
that their level of control of learning was lower than for the students in the SB context. This may explain the superiority 
of SB to VB when it comes to developing positive attitudes towards physics homework. 


Limitations 


Since the same conceptual test has been used for purposes of pre-testing, as well as for purposes of post-testing, 
there is a potential testing effect threat to the internal validity of the research experiment. Concretely, the increase in 
conceptual test scores may be alternatively explained by the fact that students memorized pre-test items and their 
solutions which helped them to solve the post-test. However, this alternative explanation is not very probable because 
students were not told in advance that they will write the same test again. They were also not given the solutions of 
the pre-test. 

Another limitation of the research experiment is related to the fact that it failed to control the amount of time 
invested in the completion of homework. However, this is mostly due to the fact that, unlike students from the PP 
group, the students from the SB and VB groups were required to make observations and/or collect measurements. 
This required extra time but obviously was not crucial for constructing target knowledge, at least when it comes 
to conceptual understanding. The fact that SB and VB students exhibited more positive attitudes towards physics 
homework, besides the fact that they invested much more time in its completion is an important finding on its own. 


Conclusions and Implications 


Much of the physics learning may happen outside the physics classroom, particularly within the homework 
context. How effective this homework learning eventually will be, depends mostly on the quality of the mere home- 
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work assignment and teacher feedback. Often, physics homework boils down to solving end-of-chapter quantitative 
problems. Such an approach does not effectively target many of the important learning goals of modern-day physics 
education. Particularly neglected are science inquiry and affective learning goals, related to the development of posi- 
tive attitudes towards school physics. On the other hand, it is widely known that students’ attitudes towards school 
physics largely affect career choices and readiness for lifelong learning of physics. Therefore, it is important to identify 
alternative homework approaches that may be combined with paper-and-pencil homework to target a wider spectrum 
of learning goals. 

Findings from this research support the idea that homework learning may result in large improvements of 
conceptual understanding if it is carefully designed to target key cognitive processes from a given domain. Thereby, 
conceptual understanding is mainly developed through the cognitive processing of collected information about physi- 
cal phenomena, rather than through the mere activity of collecting measurements and doing observations. However, 
providing the students with the opportunity to learn by observing and collecting measurements positively affects their 
attitude towards school physics. Concretely, simulation-based homework and video-based homework proved to be 
superior to the paper-and-pencil homework when it comes to developing positive attitudes. In fact, simulation-based 
homework was found to be the most effective, which may be explained by the fact that students like to have an active 
approach to learning and to control the pace of their own learning. 

A practical implication of this research is that different formats of physics homework should be combined in the 
teaching practice. An advantage of the paper-and-pencil homework is that it is relatively easy to be prepared and 
can be similarly effective to simulation-based and video-based homework when it comes to developing conceptual 
understanding and problem-solving ability. However, to fulfill important science inquiry and affective learning goals, 
the homework format should be changed from time to time. One attractive option is to assign simulation-based in- 
quiry activities as homework. Compared to the video-based approach, the simulation-based homework assignment is 
much easier prepared and more effective in developing positive attitudes toward school physics, but equally effective 
in developing conceptual understanding. When assigning simulation-based homework, it is strongly suggested to 
accompany it with carefully designed worksheets which may offer important guidance in the absence of the teacher. 
The worksheets should be designed to facilitate students’ discovery of relationships, application of these relationships 
through predict-observe-explain activities and solution of physics problems situated within the context of simulations. 

In future research, it may be interesting to explore whether the effectiveness of simulation-based homework 
may be increased through learning by collaboration. Also, it would be interesting to conduct a carefully controlled 
experiment to compare the effectiveness of simulation-based learning in the classroom and homework setting. 
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